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ABSTRACT 

In the context of higher education, the development of 
students* problem solving skills continues to be an area of much ongoing 
research. Effective teaching of problem solving requires the adoption of 
process-based approaches that reveal to students the ways that experts solve 
problems, and the coaching of students in higher order and metacognitive 
skills that lead them away from a preoccupation with finding solutions and 
towards building up a repertoire of problem solving strategies. It is 
suggested that online environments and computer resources can scaffold the 
acquisition of domain knowledge and systematic problem solving skills. This 
article acknowledges that there are multiple ways to support complex problem 
solving, and that online environments hold great promise in creating 
effective instructional interactions. This study focuses on the metacognitive 
aspects of problem solving and more particularly on those aspects of 
technology-based scaffolding that support reflection, process support, and 
the anchoring of skills to domain knowledge. (‘Contains 20 references and 4 
tables.) (Author) 
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Abstract: In the context of higher education, the development of students* problem 
solving skills continues to be an area of much ongoing research. Effective teaching of 
problem solving requires the adoption of process-based approaches that reveal to students 
the ways that experts solve problems, and the coaching of students in higher order and 
metacognitive skills that lead them away from a preoccupation with finding solutions and 
towards building up a repertoire of problem solving strategies. It is suggested that online 
environments and computer resources can scaffold the acquisition of domain knowledge 
and systematic problem solving skills. This article acknowledges that there are multiple 
ways to support complex problem solving, and that online environments hold great 
promise in creating effective instmctional interactions. This study focuses on the 
metacognitive aspects of problem solving and more particularly on those aspects of 
technology -based scaffolding that support reflection, process support, and the anchoring 
of skills to domain knowledge. 
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Problem solving skills as central to learning 

In tertiary education, there is an urgent need for professionals who can solve real problems, anticipate and 
predict problems and find realistic solutions. To meet this need effectively tertiary educators must now 
reexamine methods of teaching problem solving. In discussing pedagogical approaches, Hobden (1998; 
223) states that: 

"It (problem solving) is a routine activity occupying a large proportion of curriculum time and 
plays a central role in student’s experience of classroom life. From the first days of science 
instruction, sets of routine problem tasks assigned by the teacher have been part of classroom life. 

As a teaching strategy, they have largely been used uncritically. It would appear that nearly all 
physical science education, and especially the physics component, seems to be based on the 
optimistic assumption that success with numerical problems breeds an implicit conceptual 
understanding of science , " 

These comments apply to the way problem solving has been taught in the past, and to approaches that rely 
on traditional ‘show and tell’ where students do not engage actively wth the problem or context. The 
traditional method of showing examples of solutions to problems followed by student practice represents a 
rather primitive approach in contrast to the teaching of acting, music or a sport, where the skills required 
for competent performance are taught in context, often in real world scenarios, and integrated so that expert 
performance results. Taconis et al (2001) have recently analyzed articles appearing in international 
journals between 1985 and 1995 on the effectiveness of teaching strategies for science problem solving. 
Very briefly, this analysis showed that in student performance and achievement, knowledge of strategy and 
practice of problem solving turned out to have little effect, whereas accounts of effective teaching of 
problem solving all give attention to contextualised strategies related to mastery of domain knowledge. The 
learning conditions recognised by Taconis et al (2001) as significant for building problem solving skills 
were those which provide learners with guidelines and criteria they can use in judging their own problem 
solving processes and products. The provision of immediate feedback to learners is also essential. These 
conclusions are congruent with earlier research carried out by Alexander & Judy (1988), Clarke (1992) and 
Lajoie (1993). 
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The learning paradox 



A number of contemporary designs for learning environments require student self-direction and high-level 
metacognitive control. Exploratory and open-ended learning environments enable students to generate 
kno>vledge and engage in critical thinking. Similarly, project based approaches assume that students are 
able to generate questions and produce a final product that represent knowledge integration. Many students 
however lack essential metacognitive skills and a repertoire of learning strategies to enable them to 
maximise their learning in innovative learning environments. Table 1 shows how a range of contemporary 
learning designs assume metacognitive knowledge. 



Example 


Learning scenarios 


Metacognitive skill needed 


Anchored instruction 


Narratives, stories, real life anchors 


Capacity to define problems 
and abstract from case 


Open ended learning 
environment 


Multiple scenarios and viewpoints 


Self-direction and self 
management 


Project based learning 


Collaborative, task based learning 
environments 


Management of information, 
self and others 


Problem based learning 


Presentation of cases and events that 
present potential problems 


Capacity to identify the 
problem and select resources to 
solve it 



Table 1 : Metacognitive requirements of contemporary student -centered learning environments 

These learning environments though highly successful, assume that students are goal driven and self- 
directed. Yet, these expectations bring with them a range of assumptions, primarily that students have 
metacognitive skills to enable them to cope with self-direction learning. Other research has shown that the 
processing demands of these environments are problematic and need to be investigated. One of the issues 
of most concern is that of the learning paradox noted by Schank & Cleave (1995) '‘How can students learn 
by doing what they do, when they do not know how to do what they have to do to learn? Stated quite 
simply, project and problem based learning assumes that students can access and apply knowledge and 
metacognitive strategies and engage in self-regulated learning. It is well established that in order to learn 
effectively, a repertoire of learning strategies and the capacity to manage one's own learning are 
fundamental (Boekaerts, 2000; De Corte et al, 2000). It is this range of skills that we refer to as 
metacognition. 



The role of metacognition in learning and problem solving 

Metacognition is a learner's knowledge about his or her processes of cognition and the ability to control and 
monitor those processes as a function of the feedback the learner receives via outcomes of learning (cf 
Metcalfe & Shimamura, 1994; Schraw 1998b). Metacognitive knowledge refers to what the learner 
understands and believes about a subject matter or a task, and the judgments s/he makes in allocating 
cognitive resources as a result of that knowledge. Metacognitive control refers to the strategies the learner 
uses to achieve specific learning goals - strategies like planning aid organizing, allocating attention to 
relevant and irrelevant factors, looking for relationships and patterns, monitoring comprehension, 
identifying and testing procedures, evaluating outcomes, and reflecting on learning (cf Jacobs & Paris, 
1987). Schraw (1998a) explains that attentional resources, existing cognitive strategies, and awareness of 
breakdowns in comprehension, are all enhanced by training in metacognitive knowledge and skills. While 
there are examples of successful metacognitive instruction in the literature, the most effective ones involve 
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providing the learner with both knowledge of cognitive processes and strategies, and experience or practice 
in using them. Simply providing knowledge without experience or vice versa does not seem to be sufficient 
for the development of metacognitive control (Volet, 1991). It is also essential that learners have an 
opportunity to evaluate the outcome of their efforts, to reflect and self-assess their own approaches to 
learning. 

Four categories of metacognitive knowledge are recognised as important and affect task performance and 
achievement of distance learners (White, 1 999). These are as follows: 

□ Self-knowledge: Self-knowledge entails individual capacity to recognise their strengths and 
weaknesses and to evaluate themselves, 

□ Task-knowledge: This involves understanding the demands of tasks and what they require, 

□ Strategic knowledge: This refers to the knowledge of usefulness of strategies available for achieving 
learning goals. 

□ Knowledge of plans and goals: This refers to learner’s capacity to set and maintain goals and to 
record what they intend to do through their learning. 



Design of learning environments for metacognitive support 

In response to a perceived need to support problem solving, an on-line tutorial designed in WebCT, called 
met AHEAD, has been created for first year science students in Biology, Biophysics and Chemistry, The 
aim of the tutorial is to help students explicitly develop their metacognitive skills in science problem 
solving. The tutorial comprises four modules - an introduction to thinking and learning, knowledge maps, 
explanations and descriptions, problem solving. In developing the resource, instructional design was guided 
by current research on constructive learning, and metacognitive skills development, Lin et al (1999) in 
considering the technological design of learning environments have noted that four features are important 
for effective scaffolding of reflective thinking by students. These are process displays, process prompts, 
process models and reflective social discourse. The first module of met AHEAD introduces students to key 
aspects of learning and problem solving and allows them to rate their own problem solving skills on a 
standard metacognitive inventory test. The second module helps them to build concept maps for topics in 
their subject areas. The last two modules concentrate on the sorts of skills needed for explanations and 
descriptions (as most often needed in Biology) and quantitative problems (more often needed in Biophysics 
and Chemistry). 

As students tackle problems in the modules they are asked to engage in a number of the following 
activities: predict their own expected success on problems; discuss their success with other students during 
and after attempting the problem; explicitly consider strategies for use with particular problems; examine 
other students’ answers and comments on them; listen to audio clips, view video clips or read transcripts of 
other students as they worked on problems and comment on these; rate other students' answers; post their 
answers to a bulletin board for discussion with other students; view lecturer's model answers and comment 
on them. 

Students maintain an on-line logbook to record their responses to the various prompts throughout the 
tutorial and to record their answers to problems. In metAHEAD particular attention has been paid to 
supporting the four categories of knowledge identified as essential for development of metacognitive skills 
(White, 1999), Process prompts and process models are in evidence in the learning activities, which 
precede the solution of example problems, and in the many worked answers of students and lecturers and 
comments thereon. Bulletin board discussion of strategies and problem solving skills together with the 
creation of collaborative teams provide opportunities for reflective social discourse. 



Evaluation of metAHEAD 

It was intended to carry out a formative, integrative evaluation of metAHEAD in order to improve the 
design and to refine various aspects of the resource. Oliver (2000) has remarked that while a range of 
methodologies exists, each may be restricted in its use and in the range of situations it can be applied to. 
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From our pragmatic perspective, we found the Open University model as described by Jones et al (1996) a 
useful framework. This approach focuses on three main themes: context, interaction and outcomes. This is 
outlined in table 2, and shows that range of data that was collected and analyzed. As practitioners, we 
sought an evaluation approach that was broad and flexible enough to suit our situation, which was the 
introduction of an innovative resource within a university context . From the outset, data has been gathered 
on the design and use of metAHEAD from pilot studies, practitioners' opinions, instructional designers who 
have used and developed the program and academic staff who have used sections in their teaching. It was 
important to include this data in our evaluation, as we needed a holistic picture of its benefits. 





Context 


luteractions 


Outcomes 


Rationale 


Need for metAHEAD at 
UNE; curriculum context 


Need to look at student 
interactions with the 
resource. 


Learning outcomes; problem solving 
outcomes; changes of perception and 
attitude must be considered 


Data 


Designers aim; principles 
underpinning design; 
pressing iterations 


Records of students' 
interactions, student 
diaries, and online logs 


Measures of effective problem 
solving, changes in attitude, strategy, 
perception of self 


Methods 


interviews; written records; 


Observations, videos, 
diaries, computer records, 
product data generated by 
students 


Focus groups, tests and questionnaires 



Table 2: Features of the evaluation approach adopted 



As metAHEAD is embedded in Web CT, it allows for the gathering of the following types of data for the 
evaluation of metacognition, data collection was relatively standard self-rating quiz data taken at beginning 
and end of semester; students' self predictions of level of success with particular problems and reflection 
upon these on completion of the problem; on-line log book entries including students' notes on strategy use 
and actual problem solutions; bulletin board discussions. These data are complemented by further data 
gathered from small face-to-face focus group discussions. As a further elaboration of our evaluation 
approach, we collected a range of data relating to the interactive and scaffolding features of metAHEAD. 
These are summarised in table 3. 



Data source 


Type of evidence 


Journal entries 


Progress logs (process data) 


Oral discussion 


Online transactions (process data ) 


Learner’s self assessment 


Prediction of success (process data) 


Solution logs 


Solutions to problems (product data ) 



Table 3: Evaluating metacognition in online environments 



Evaluation and data analysis 

Data has been gathered from focus groups giving feedback on technical issues and development of problem 
solving skills. There are two main areas of feedback gained from this evaluation. Students have greatly 
appreciated the availability of other students’ answers and particularly the comments on them. This has 
allowed them to put their own answers into a better perspective and to give a clearer idea of what the 
lecturer expects. (The fact that assessment and lecturers' expectations drive the majority of students cannot 
be escaped.) These do provide process models and support reflection by the students on many aspects of 
problem solving. Coincidentally, other student answers help some students to feel more of a part of a 
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group and that they are ’’not so stupid after all”, which can affect student motivation to succeed along with 
others. Students have identified that their motivation often parallels their success in the subject. Becoming 
part of a community of students working on a unit is particularly important for distance education students, 
who may be otherwise rather isolated. The ability to discuss issues on the bulletin board has been helpful 
for these students. 

Students have also mentioned that planning and analysis of problems, whereby parts can be tackled step by 
step has been helpful to them. Becoming more aware of and practicing such skills has been beneficial for 
students who in the past may have been daunted by a problems and given up too soon when it seemed too 
difficult. While we have provided some models and coaching online, students appear to need a more staged 
approach, with step-by-step examples and support. This ‘cognitive apprenticeship’ approach is in tune with 
the constructivist principles underpinning the design of met AHEAD, Table 4 shows a summary of student 
comments gained from the evaluation. 



Comments on what students gained from metAHEAD 

Helps think about level of confidence and predict degree of success. 

Helps motivation somewhat. 

Helps with ways to understand the question better in order to tackle it successfully. 

Helps with breaking up tasks - something they didn't commonly consider explicitly. 

Talking to other students is helpful, hear other views. 

Getting feedback from lecturer, hearing them explain and comment on lecturer's answers. 

Appreciated other students’ answers, so they could compare their own answers, and lecturer's "metacomments” on these were 
important. 

Other students' answers also show other ways of solving problems. 

Information about learning and thinking processes - eg chunking information. 

Comments on limitations of metAHEAD 

Can't replace face to face. Need personal tutor for motivation. 

Notebook tool has great technical deficiencies particularly for science answers involving formulas and equations. 

Needs to help students with more practice. 

Needs more models and demonstrations of how to work through problems step by step. 

Technical difficulties. Eg Plug-in for Quicktime movies. 



Table 4: Summary of comments from evaluation data 

Implications for instructional design 

The aim of metAHEAD was to design a learning resource to offer students support in metacognition and 
problem solving and a learning experience that was different from traditional teaching in first year 
(freshman) study. The focus was on building student self-awareness of metacognition and strategy 
development infused into disciplinary content (chemistry, physics and biology). Our initial evaluation has 
shown both positive and negative aspects of the resource, and insights gained will be used to design the 
resources and improve its capacity to support problem solving. Our choice of evaluation approach has been 
effective and worthwhile, providing valuable data on context, interactions and outcomes of the resource. 
Continuing evaluation \m 11 assist in refining the resource, tailoring to student needs and improving its 
relatedness the context in which it is being used i,e, the support of metacognition at tertiary level. 
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